Research Synthesis as a Scientific Process

The moment we are introduced to science we are told it is a cooperative, cumulative enterprise. Like the artisans who construct a building from blueprints, bricks, and mortar, scientists contribute to a common edifi ce, called knowledge. Theorists provide our blueprints and researchers collect the data that are our bricks. To extend the analogy further yet, we might say that research synthesists are the bricklayers and hodcarriers of the science guild. It is their job to stack the bricks according to plan and apply the mortar that makes the whole thing stick. Anyone who has attempted a research synthesis is entitled to a wry smile as the analogy continues. They know that several sets of theory-blueprints often exist, describing structures that vary in form and function, with few a priori criteria for selecting between them. They also know that our data-bricks are not all six-sided and right-angled. They come in a baffl ing array of sizes and shapes. Making them fi t, securing them with mortar, and seeing whether the resulting construction looks anything like the blueprint is a challenge worthy of the most dedicated, inspired artisan. Scientifi c literatures are cluttered with repeated studies of the same phenomena. Multiple studies on the same problem or hypothesis arise because investigators are unaware of what others are doing, because they are skeptical about the results of past investigations, or because they wish to extend (that is, generalize or search for infl uences on) previous fi ndings. Experience has shown that even when considerable effort is made to achieve strict replication, results across studies are rarely identical at any high level of precision, even in the physical sciences (see Hedges 1987). No two bricks are exactly alike. How should scientists proceed when results differ? First, it is clear how they should not proceed: they should not pretend that there is no problem, or decide that just one study (perhaps the most recent one, or the one they conducted , or a study chosen via some other equally arbitrary criterion) produced the correct fi nding. If results that are expected to be very similar show variability, the scientifi c instinct should be to account for the variability by further systematic work. The American Psychological Association's PsycINFO reference database defi nes a literature review as " the process of conducting surveys of previously published material " (http://gateway.uk.ovid.com). Common to all defi nitions of literature …